Texture compression techniques

ABSTRACT

A texture compression method is described. The method comprises splitting an original texture having a plurality of pixels into original blocks of pixels. Then, for each of the original blocks of pixels, a partition is identified that has one or more disjoint subsets of pixels whose union is the original block of pixels. The original block of pixels is further subdivided into one or more subsets according to the identified partition. Finally, each subset is independently compressed to form a compressed texture block.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.11/513,190, filed Aug. 31, 2006, which is incorporated by reference asif fully set forth.

FIELD OF INVENTION

The invention relates generally to field of texture compression anddecompression in computer graphics.

BACKGROUND

A texture is a one-, two- or multi-dimensional array of data items usedin the calculation of the color or appearance of fragments produced byrasterization of a computer graphics scene. A texture could be imagedata (either photographic or computer generated), color or transparencydata, roughness/smoothness data, reflectivity data, etc. Providingrealistic computer graphics typically requires many high quality,detailed textures. Providing such textures can tax available computermemory and bandwidth. Texture compression can therefore significantlyreduce memory and bandwidth requirements.

Texture compression has thus become a widely accepted feature ofgraphics hardware in general and 3D graphics hardware in particular.Indeed, forms of texture compression date back to the earliest pieces ofgraphics hardware. The goal of texture compression is to reduce storageand bandwidth costs on the graphics system while retaining as much ofthe quality of the original texture as possible. Attaining this goal,however, has proved to be complex and has generated several differentapproaches, some of which are described briefly below.

Palletization is amongst the oldest forms of texture compression. Itsroots date back to the earliest days of color computer graphics.Typically, palletized formats are represented by 8 bits of data perpixel, permitting a maximum of 256 colors chosen from the completecolorspace (usually quantized to 16 or 32 bits). Some images can be wellrepresented by this approach, but it is not uncommon for the palletizingprocess to generate significant visual artifacts. Palletization isclearly limited when dealing with real-world images such as photographs,where the limited set of available colors is quickly exhausted. Imagespace techniques such as dithering are used for improving the quality ofpalletized images, but are difficult to use with textures because if thetexture is magnified, the desired effect of the dithering may be lost,and the dithering itself may introduce undesirable artifacts. Palletizedmethods have some additional attributes that can make them lessattractive for implementation in graphics hardware—for instance, theyintroduce an indirection when looking up color values. Palletizationmight also require storage for multiple palettes simultaneously formulti-texturing support. Generally, the quality achieved per-bit isquite low with palletization, and it has largely been superseded by moreadvanced methods.

Vector Quantization (“VQ”), developed by PowerVR, is another specifictexture compression technique. It works by storing a limited “codebook”of representative entries to define a texture. The codebook entries areblocks of pixels of some size (typically 2×2 or larger). For each blockof pixels in the original texture, an index is stored to the codebookentry that most closely approximates the block. VQ can achieve very highcompression rates (down to about 2 bits per pixel) while still retainingfair quality. Nonetheless, it shares some of the undesirable qualitiesof palletization with respect to texture compression. For instance, thetype of artifacts introduced by VQ compression can be quite noticeableon texture images, and it frequently shows visible artifacts on somecommon texture contents such as smooth gradients.

The Joint Photographic Experts Group (JPEG) algorithms are another imagecompression technique. JPEG achieves a very high quality of compressionat a low bit rate, but the compression is of a variable rate. Variablerate compression makes addressing the texture map very difficultcompared to fixed-rate schemes. As a result, there has been no adoptionof JPEG compression in consumer 3D graphics systems except for thelimited purpose of reducing a system memory imprint. For example, JPEGcompression is used on Sony's Playstation 2 to reduce the system memoryfootprint, but the system does not texture directly from the compressedJPEG representation.

DXTC (sometimes referred to as DXTn) is a block-based texturecompression scheme has been adopted by all major hardware vendors and isthe most widely used today. An extension of Block Truncation Coding(BTC), it explicitly stores two 16-bit colors per 4×4 pixel block andtwo other colors that are implicitly represented as being interpolantsbetween these endpoints, with an index of 2 bits per pixel to choose thecolors for the pixel block. As a result it achieves overall colorcompression to 4 bits per pixel. DXTC represents the original texturedata quite well in the majority of cases. However, DXTC has problemswith textures having many different color hues within each block.Additionally, the low precision of the endpoints and small number ofinterpolants can produce some noise on gradients, particularly ones thatare oriented diagonally to the pixel blocks. DXTC also has problems withtextures containing blocks that have multiple separate color gradientsat different orientations, as accurate compression of one gradient musttypically be sacrificed when mapping the points to a line through thecolorspace. This happens frequently in images such as normal maps. ADXTC extension allows 4 component images (with alpha) to be representedat 8 bits per pixel.

FXT 1 is a competing compression scheme with DXTC. It essentiallyextends DXTC with some additional block types that can be mixed withinany given image, and also provides a 4 bits per pixel compression modefor textures with alpha. The gains in image quality over DXTC were neverconclusive and FXT 1 received limited industry support.

PVR-TC is a recently developed compression scheme that scales an imagedown to a fraction of its original size and then scales it back up toobtain a good first-approximation of the original image. Texturecompression is thus achieved by storing the downscaled version and thenadding some modulation to create a fairly accurate reconstruction of theoriginal data. This texture compression scheme works well for some typesof data (particularly smooth gradients), and scales reasonably even tolow bit rates. However, PVR-TC has a tendency to blur images somewhatand lose high frequency details. Occasionally the PVR-TC compressionalso seems to introduce other artifacts such as high frequencymodulation noise and ringing.

A review of current texture compression techniques and their limitationsreveals a need for improvements. An ideal solution would allowcompression of various types of data and would have the flexibility tomake the best use of available memory and bandwidth. Additionally,traditional texture compression schemes target a specific type oftexture content (e.g., JPEG for photographic images) and perform wellwithin that set, but perform poorly as soon as presented with a type ofimage outside of the designated set. Another challenge is thus tobroaden the scope of texture compression to adequately cover a widerbase of image types.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and form a partof the specification, illustrate the present invention and, togetherwith the description, further serve to explain the principles of theinvention and to enable a person skilled in the pertinent art to makeand use the invention.

FIG. 1 illustrates an exemplary system in which the describedembodiments may operate.

FIG. 2 illustrates the flow of texture data through an exemplary system.

FIGS. 3A-3C are flow charts illustrating a compression method.

FIG. 4 illustrates various partitions.

FIG. 5 is a flow chart illustrating a decompression method.

FIGS. 6A-6D illustrate an exemplary set of 64 partitions, each havingfive disjoint subsets.

FIG. 7 illustrates an exemplary set of 32 partitions, each having twodisjoint subsets.

FIG. 8 illustrates an exemplary set of 12 partitions, each having threedisjoint subsets.

The present invention will now be described with reference to theaccompanying drawings. In the drawings, like reference numbers indicateidentical or functionally similar elements. Additionally, the left-mostdigit(s) of a reference number identifies the drawing in which thereference number first appears.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Introduction

Textures are one-, two-, or multi-dimensional data arrays. Textures aresometimes used to enhance or change the appearance of surfaces ingraphics. A “texel” is a texture element in the same way a “pixel” is apicture element. The terms “pixel” and “texel” may be used synonymouslyherein and simply refer a discrete unit of data—e.g., data at an (x, y)location in a frame buffer, texture buffer or other type of memory orarray. The compression and decompression methods described herein may beused to compress various types of texture information including imagedata, picture data, transparency (alpha) information, smoothness orroughness data, or any other similarly structured data. As such, theterm texture is used broadly herein to refer to the data beingcompressed or decompressed using the described methods.

Texture mapping is the process of applying a texture to a fragment,pixel or primitive representing a surface to be displayed. Texturemapping is often used to add realism to a scene. For example, one canapply a picture of a building façade to a polygon representing a wall.During texture mapping, a compressed texture element—e.g., a block oftexels—is obtained from texture memory and represents the color or othersurface characteristic of a corresponding fragment or pixel on thesurface to be displayed.

As hardware and software graphics platforms support increasinglysophisticated texture mapping techniques, the scenes that developerswould like to render have grown much more complex. Complex scenes thusoften make greater use of textures. One of the costs of texture mappingis that the texture images often require a large amount of memory.Memory for textures can be limited, especially in hardware renderingsystems where textures are placed in a dedicated memory in the graphicssubsystem. Texture compression, which reduces the amount of data neededto store texture map images, allows a user to fit more texture data intoavailable memory. The result is an ability to use more textures forhigher accuracy and increased rendering quality.

There exist both “lossy” and “lossless” texture compression schemes. Alossy texture compression scheme refers to a compression technique whereit is impossible to exactly recreate the texture duringdecompression—i.e., the original texture data is changed irretrievably.A lossless texture compression scheme, on the other hand, refers tothose techniques that permit the exact recreation of the originaltexture—i.e., the original texture data can be identically recovered bydecompressing the texture compressed by a lossless compressiontechnique. The present invention typically implements lossy texturecompression techniques.

Lossy texture compression techniques seek a balance between texturecompression ratio (i.e., the amount of compression) and image quality. Ahigh texture compression ratio provides benefits with respect to memoryuse and speed, but typically sacrifices image quality. On the otherhand, textures may be stored at a high level of quality, but typicallyat the cost of reduced compression ratios. It will be appreciated bythose skilled in computer graphics that certain applications may valueimage quality, while others may value a high texture compression ratio(i.e., a low bit-per-pixel number). The overall goal, therefore, is tomake the most efficient use of available resources while meeting theneeds and limitations of the application, the user, and the user'sgraphics system.

As the expectations for higher levels of image quality in computergraphics increase, the limitations of DXTC and other lossy texturecompression methods described above are becoming more apparent. Giventhat the amount of available memory and bandwidth is constantlyincreasing the present invention permits increased image quality with alower compression ratio as well higher quality compression than existingmethods at the same compression ratio. When going to a higher number ofbits-per-pixel, one challenge is trying to achieve the same quality perbit as the schemes with higher compression ratios. To address thischallenge, the present invention further refines block-based imagecompression techniques such as DXTC with a flexible method forpre-compression partitioning of texture blocks. The pre-compressionpartitioning enhances image quality by flexibly selecting partitionssuitable to the characteristics of the texture block being compressed,rather than adopting the one-size-fits-all approach suggested in othermethods.

Sample Environment

Before describing embodiments of the present invention in detail, it ishelpful to describe an example environment in which the data compressiondevice may be implemented. FIG. 1 illustrates a graphics system 100. Thesystem may include, but is not limited to, a computer 110, various inputdevices 160 such as a keyboard or a mouse (not shown), and variousoutput devices 170 such as a liquid crystal display (LCD) monitor or acathode ray tube (CRT) monitor (not shown). The computer 110 mayinclude, but is not limited to a central processing unit (CPU) 120, agraphics processing unit (GPU) 140, a main memory 130, and a graphicsmemory 150. As will be understood by those having ordinary skill in theart, the components may be combined in various ways. For example, theCPU and the GPU may be combined into a single device. Similarly,exemplary embodiments of the invention may combine main memory andgraphics memory into a single memory. Other combinations are similarlypossible.

The computer 110 may be a portable computer, a laptop computer, adesktop computer, server, mainframe, handheld device (e.g., mobilephone, camera, portable media player), digital television or the like.The main memory 130 may be random access memory (RAM) or some otherstorage device. For static data, the main memory may also contain readonly memory (ROM). Main memory 130 may be used for storing informationduring execution of instructions by the CPU 120 such as processorinstructions, temporary variables, or cached information. Graphicsmemory 150 may also consist of RAM, ROM or other storage and isprimarily used for storing graphics information such as sceneinformation and texture data. Also illustrated is an external memory orstorage device 135, such as a hard disk or other storage device, thatcan be used to store texture data. It should be noted that textures maybe stored in uncompressed or compressed format. Pre-compressed datawould pass through the CPU unmodified.

The graphics system 100 may also include an external communicationdevice 180 such as a modem, a wired or wireless network interface card,or other well-known interface devices. External communication device 180allows a communication link to a local area network (LAN), wide areanetwork (WAN), the Internet or any other well-known type of externalnetwork. All the elements of the graphics system 100 are typicallycoupled to a communications bus 190 through which information andinstructions are routed.

The GPU 140 contains the graphics hardware, which typically includes arendering pipeline for transforming scene and texture data intoinformation ready for display. Some graphics hardware systems mayinclude a separate pipeline that is dedicated to texture information.The GPU 140 also typically has its own local storage that holds theinformation to be output to a visual display device.

FIG. 2 further illustrates the path texture data may take through anexemplary graphics system. An uncompressed texture source 210 (e.g.,texture database or library) is coupled to CPU 120. Within CPU 120 is atexture compression engine 220. The texture compression engine 220 isprimarily responsible for implementing the texture compression methodsof the present invention using hardware, software or a combination ofboth.

A compressed texture may then be stored in graphics memory 150 orexternal memory 135, which are coupled to CPU 120. Texture data for useby the GPU 140 is stored in the graphics memory 150. In someimplementations this may be a dedicated region of memory—i.e., texturememory 250. The graphics memory 150 and CPU 120 are both coupled to GPU140. Within GPU 140 is a texture decompression engine 240. Texturedecompression engine 240 is primarily responsible for decompression ordecoding compressed texture data such that it can be used by GPU 140.The GPU may use this texture data to produce data used by the outputdevice 170 or in further processing.

Compression Principles

As noted above, the present invention represents a further refinement ofblock-based texture compression schemes such as DXTC. Like DXTC, thebasic compression algorithm implemented in a preferred embodimentdivides or splits the image up into discrete blocks of pixels (e.g., 4×4pixel or 8×8 pixel blocks) and compresses them.

However, rather than mechanistically compressing the regular textureblocks, the present invention introduces the idea of further subdividingor partitioning the block internally into “subsets” of different shapeand size before compression. Each subset is then compressed in a mannersimilar to DXTC. Precompression subdivision addresses several imagequality problems with existing DXTC compression, and results in asignificant overall increase in perceived and measured quality.Furthermore, it provides improvement in the ability to compress lesscommon types of texture data (such as non-photorealistic textures,cartoons, etc.).

FIGS. 3A-C are flowcharts more specifically illustrating one embodimentof the present invention. Initially, an original texture to becompressed is received in a texture compression engine. As with otherblock-based texture compression schemes, the original texture is thensplit or divided into original blocks of pixels according to step 305.Typically, the original block of pixels is a square or rectangularblock—e.g., 4×4 or 8×8 pixel. The size of the original block of pixelsis variable, and may be selected according to user needs, applicationdemands and/or system capabilities.

As indicated in 310, each original block of pixels is then subjected tosteps 315 through 325. According to step 315 a partition is identifiedthat describes the manner in which the original block of pixels is to befurther split. Each of the partitions define a plurality of disjointsubsets of pixels whose union is the original block of pixels. Apartition can consist of a single subset, in this case the subset isidentical to the single block of pixels. FIG. 4 illustrates some basicexamples of the types of partitions and the subsets they contain. Thesize and arrangement of the subsets within a partition is completelyvariable—i.e., there is no restriction considering the number orconfiguration of the subsets except for the number of bits dedicated todefining them. For example, a partition may have 8 or 16 subsets as inpartitions 402 and 404, respectively. Alternatively, a partition mayhave 3 or 6 subsets as in partitions 408 and 406, respectively.

Typically, the partition is selected from a pre-defined set ofpartitions. The number of available partitions in the pre-defined setmay vary, but is limited by the number of bits in the data structurededicated to relaying that information. For example, if there are 6“partitionBits” dedicated to describing the partition, then thepre-defined partition set could contain up to 64 different partitions.Some embodiments could have several sets of partitions, the partitionsin each set divide the block into the same number of subsets, with theset of partitions to be used (and hence the number of subsets for theblock) being identified by a selector. For example, if the selector hastwo bits, and there are 6 partition bits, then there could be one set of64 two-subset partitions, another set of 64 three-subset partitions, athird set of 64 four-subset partitions, and a fourth set of 64five-subset partitions. The specific data structure for this embodimentis described in more detail below.

An exemplary set of 64 partitions, each having 5 subsets, is illustratedin FIGS. 6A-6D. For clarity, the complete set of partitions isillustrated over 4 pages of figures, with 16 such partitions illustratedby partition 600A in FIG. 6A. An exemplary partition 602 is shown inFIG. 6A. Therein, an 8×8 pixel block has been subdivided into 5 disjointsubsets with a first subset containing those pixels labeled 0, a secondsubset containing those pixels labeled 1, a third subset containingthose pixels labeled 2, a fourth subset containing those pixels labeled3, and a fifth subset containing those pixels labeled 4.

Similarly, FIG. 7 illustrates 32 partitions, each having two disjointsubsets. An exemplary partition 702 is illustrated in FIG. 7. Therein, a4×4 pixel block 702 has been subdivided into two disjoint subsets with afirst subset containing those pixels labeled 0 and a second subsetcontaining those pixels labeled 1. FIG. 8 illustrates a set of 16partitions, each of which has been subdivided into three disjointsubsets. An exemplary partition 802 is illustrated in FIG. 8. Therein, a4×4 pixel block 802 has been subdivided into 3 disjoint subsets with afirst subset containing those pixels labeled 0, a second subsetcontaining those pixels labeled 1, and a third subset containing thosepixels labeled 2.

The partition set could be of arbitrary size up to the combinatoriallimit of the number of ways to partition a block. In embodimentsdesigned for efficient encoding a more limited set of partitions will bedefined in order to fit within the desired code size. An embodimentcould allow a unique partition set for each texture, or it could requireall textures to share the same fixed partition set. It is possible toevaluate a quality function for each original block of pixels to aid inselecting an appropriate partition. Once a partition has beenidentified, the original block of pixels is subdivided into one or moresubsets according to the identified partition, as illustrated in step320.

Next, according to step 325, each subset is then independentlycompressed to form a compressed texture block. In an embodiment, each ofthe subsets may be compressed using two explicit endpoint parametersthat define a line in the colour space, and a plurality of implicitpoints on the line selected according to a predefined quantization.However, the described methods are not limited to this particularcompression scheme. For example, palletization or other vectorquantization compression algorithms could be implemented as well.

After the subsets of pixels have been compressed, the compressed textureblock may be evaluated to further refine partition selection. Two suchevaluation embodiments are illustrated in FIGS. 3B and 3C. Evaluationtypically requires (i) decompression of the compressed texture block toobtain an approximation of the original block of pixels, and (ii)comparison of the original block of pixels and the approximation of theoriginal block of pixels to obtain a quality parameter. These steps arereflected in steps 330 and 332 of FIG. 3B, and steps 340 and 342 of FIG.3C.

The quality parameter is derived from a predefined function used toquantify a certain aspect of the decompressed texture block as itcompares to the original block of pixels. For instance, the predefinedfunction could evaluate the pixel colors in the original block of pixelsas compared to the pixel colors in the approximation of the originalblock of pixels to derive an overall error approximation. An errorapproximation may be a root-mean-square (RMS) error composed of, forexample, the squared sum of individual pixel color error contributionsover all the pixels in the block. Alternatively, other functions may bedevised, depending on the users needs. For example, the function couldevaluate luminance, or transparency, or any other texture parameterdeemed important by a user. As detailed below, the quality parameter isused to measure the relative success of the compression for the selectedpartition against other partitions, or against a pre-selected qualityparameter threshold.

In the embodiment of FIG. 3B, a decision is made in step 334 as towhether the quality parameter for the identified partition meets apre-selected threshold. If it does, then the identified partitionbecomes the final partition for the current block of pixels according tostep 336. If the quality parameter does not meet the pre-selectedthreshold, then a determination is made according to step 335 as towhether partitions remain for which no quality parameter has beenobtained. If such an untested partition remains, then the method returnsto step 315. In this embodiment, a user can select a minimum standard ofquality for the texture block compression based on a variety ofparameters. In this embodiment, the quality parameter values are alsotracked so that, if the threshold quality value is not achieved, thenthe partition yielding the best quality value can be chosen, asillustrated in step 337.

Next, in step 338, the original block of pixels is subdivided into oneor more final subsets according to the final partition. The subsets fromthis final subdivision are then independently compressed to form acompressed texture block, as illustrated in step 339.

In an alternative embodiment, represented by FIG. 3C, the partition isidentified from a predefined set of partitions. Then, a qualityparameter is obtained for each partition in the set of partitions.According to step 344 of FIG. 3C, if a quality parameter has not beenobtained for each partition in the predefined set of partitions, themethod returns to step 315, where another partition is selected. In thisembodiment, the system then selects the partition yielding an optimalquality parameter, as described in step 346. Next, in step 348, theoriginal block of pixels is subdivided into one or more final subsetsaccording to the final partition. The subsets from this finalsubdivision are then independently compressed to form a compressedtexture block, as illustrated in step 349.

One of skill in the art could devise various other means for identifyingand selecting an appropriate partition. Such embodiments could betailored for the specific needs of the programmer, and depend on theparticular implementation for which data compression is desired.

Decompression Principles

Decompression of texture blocks that have been compressed according tothe present methods involves essentially working backwards. FIG. 5 is aflowchart illustrating such a decompression scheme. According to step505, a compressed texture block is received that represents an originaltexture block.

The next decompression step 510 involves determining the partition thatwas used for the compressed texture, the partition defining one or moredisjoint subsets into which the compressed texture block is to beunpacked. As noted above in the compression discussion, the partitiondefines the manner in which the uncompressed texture blocks were furthersubdivided into disjoint subsets prior to compression. Thus, the sizeand shape of the subsets must be determined before decompressing thesubsets.

According to step 515, the data for each subset is then unpacked basedon the determined partition. “Unpacking” refers to extracting theinformation from the block that is required to decompress the datastored in the subset. Finally, according to step 520, the subsets areindependently decompressed to generate an approximation of the originaltexture block. Typically, the approximation would then be used by theGPU's texture mapping algorithm and applied to a rendered surface fordisplay, but in some cases it could instead be used in further stages ofprocessing.

Exemplary Embodiments

Described below are several exemplary embodiments of the above describedcompression methods. They include exemplary data structures and pseudocode for accomplishing the compression principles described above. Theinvention is not limited to these embodiments, but only by the scope ofthe appended claims. The skilled artisan could envision and implementvariations on these embodiments without departing from the spirit andscope of the invention.

A first embodiment is a direct, higher quality replacement for DXTCcompression. It compresses 24-bit RGB data to 4 bits per pixel.Punch-through alpha encoding from DXTC is also supported, where one ofthe indices in the data structure supports a fully transparent“black-pixel” for so-called “punch-through” textures. As illustratedbelow, the texture is divided into blocks of 8×8 pixels for compression.Each compressed block has a notional format described below. However,the exact details of the packing and storage will be implementationdependent, and could be reordered to make the hardware decompression assimple as possible. In this first embodiment, a texture block issubdivided into two, three, four or five subsets. The particular datastructures are described more fully below.

Partition Data Structure with Two Subsets:

  typedef union {   struct   {  unsigned int blockType : 2;  unsignedint colour0:14, colour1:14, colour2:14, colour3:14;  unsigned intpartitionBits : 6;  unsigned int t00:3, t01:3, t02:3, t03:3, t04:3; unsigned int t05:3, t06:3, t07:3, t08:3, t09:3;  unsigned int t0a:3,t0b:3, t0c:3, t0d:3, t0e:3;  unsigned int t0f.:3, t10:3, t11:3, t12:3,t13:3;  unsigned int t14:3, t15:3, t16:3, t17:3, t18:3;  unsigned intt19:3, t1a:3, t1b:3, t1c:3, t1d:3;  unsigned int t1e:3, t1f:3, t20:3,t21:3, t22:3;  unsigned int t23:3, t24:3, t25:3, t26:3, t27:3;  unsignedint t28:3, t29:3, t2a:3, t2b:3, t2c:3;  unsigned int t2d:3, t2e:3,t2f:3, t30:3, t31:3:  unsigned int t32:3, t33:3, t34:3, t35:3, t36:3; unsigned int t37:3, t38:3, t39:3, t3a:3, t3b:3;  unsigned int t3c:3,t3d:3, t3e:3, t3f:3; } two PartitionMode;

Partition Data Structure with Three Subsets:

struct {  unsigned int blockType : 2;  unsigned int colour0:20,colour1:20, colour2:20;  unsigned int colour3:20, colour4:20,colour5:20;  unsigned int partitionBits : 6;  unsigned int t00:2, t01:2,t02:2, t03:2, t04:2;  unsigned int t05:2, t06:2, t07:2, t08:2, t09:2; unsigned int t0a:2, t0b:2, t0c:2, t0d:2, t0e:2;  unsigned int t0f:2,t10:2, t11:2, t12:2, t13:2;  unsigned int t14:2, t15:2, t16:2, t17:2,t18:2, t19:2;  unsigned int t1a:2, t1b:2, t1c:2, t1d:2, t1e:2, t1f:2; unsigned int t20:2, t21:2, t22:2, t23:2, t24:2, t25:2, t26:2, t27:2; unsigned int t28:2, t29:2, t2a:2, t2b:2, t2c:2, t2d:2, t2e:2, t2f:2; unsigned int t30:2, t31:2, t32:2, t33:2, t34:2, t35:2, t36:2, t37:2; unsigned int t38:2, t39:2, t3a:2, t3b:2, t3c:2, t3d:2, t3e:2, t3f:2;  }threePartitionMode;

Partition Data Structure with Four Subsets:

struct {  unsigned int blockType : 2;  unsigned int colour0:15,colour1:15, colour2:15, colour3:15;  unsigned int colour4:15,colour5:15, colour6:15, colour7:15;  unsigned int partitionBits : 6; unsigned int t00:2, t01:2, t02:2, t03:2, t04:2, t05:2, t06:2, t07:2; unsigned int t08:2, t09:2, t0a:2, t0b:2, t0c:2, t0d:2, t0e:2, t0f:2; unsigned int t10:2, t11:2, t12:2, t13:2, t14:2, t15:2, t16:2, t17:2; unsigned int t18:2, t19:2, t1a:2, t1b:2, t1c:2, t1d:2, t1e:2, t1f:2; unsigned int t20:2, t21:2, t22:2, t23:2, t24:2, t25:2, t26:2, t27:2; unsigned int t28:2, t29:2, t2a:2, t2b:2, t2c:2, t2d:2, t2e:2, t2f:2; unsigned int t30:2, t31:2, t32:2, t33:2, t34:2, t35:2, t36:2, t37:2; unsigned int t38:2, t39:2, t3a:2, t3b:2, t3c:2, t3d:2, t3e:2, t3f:2;  }fourPartitionMode;

Partition with Five Subsets:

  struct {  unsigned int blockType : 2;  unsigned int colour0:12,colour1:12, colour2:12, colour3:12;  unsigned int colour4:12,colour5:12, colour6:12, colour7:12;  unsigned int colour8:12,colour9:12;  unsigned int partitionBits : 6;  unsigned int t00:2, t01:2,t02:2, t03:2, t04:2, t05:2, t06:2, t07:2;  unsigned int t08:2, t09:2,t0a:2, t0b:2, t0c:2, t0d:2, t0e:2, t0f:2;  unsigned int t10:2, t11:2,t12:2, t13:2, t14:2, t15:2, t16:2, t17:2;  unsigned int t18:2, t19:2,t1a:2, t1b:2, t1c:2, t1d:2, t1e:2, t1f:2;  unsigned int t20:2, t21:2,t22:2, t23:2, t24:2, t25:2, t26:2, t27:2;  unsigned int t28:2, t29:2,t2a:2, t2b:2, t2c:2, t2d:2, t2e:2, t2f:2;  unsigned int t30:2, t31:2,t32:2, t33:2, t34:2, t35:2, t36:2, t37:2;  unsigned int t38:2, t39:2,t3a:2, t3b:2, t3c:2, t3d:2, t3e:2, t3f:2;  } fivePartitionMode; }

With respect to the above described exemplary data structures, the<blockType> field has 2 fixed bits that indicate how many subsets are inthe partition. The next field indicates the color endpoints for thesubsets. In each case there are 2 colors stored explicitly per subset,so a partition having 2 subsets stores 4 colors, and a partition havingfive subsets stores 10 colors. Each data structure has 6 “partitionBits”that are used to choose a partition from a pre-defined set of 64possible partition modes. Each <blockType> has its own set of 64possible pre-defined partitions.

In practical terms, in order to simplify the decompression and make itcheaper, this first embodiment uses only one base set of 64 explicitlypre-defined partitions, defined for the 5-subset case, as illustrated inFIGS. 6A-D. One can then, for example, create tables for the partitionswith fewer subsets by merging regions within this table—i.e., performingunions on the subsets within a partition. For example 4 subsets could bederived by merging subsets 4+3 together, while 3 subsets could bederived by merging subsets 4+3 together, and 0+1 together. Othercombinations are, of course, possible. By deriving the partitions forall modes from the above table we can reduce the storage requirementsfor the tables in hardware.

Preferably, the colors for the partition subsets are reconstructed asfollows: Each subset uses two of the explicitly defined colors that aredirectly mapped to the subset. In this embodiment the explicit colorsare promoted from their base representation up to 8 bits-per-componentby shifting and replication of the high bits to the low bits. That is, 8bits each for Red, Green and Blue (“RGB 8.8.8”) in what is referred toas “RGB” color space.

Colors are then generated for the subset by using the two explicitcolors as the endpoints of a line in RGB color space. The remaining(implicit) colors are evenly distributed along that line. For higherquality in this embodiment it is desirable for the implicit colors to bederived at higher precision than the endpoints—ie. more than 8bits-per-component. Each pixel or texel in the subset has an index thatlooks up which color to use from the line. Each subset can use one oftwo different distributions of the implicit colors relative to theendpoints along the line. We will, in future, refer to these differentdistributions as colour ramps—the ramp consists of the endpoints and thedistributed colours. Which ramp to use is decided by treating theendpoints as unsigned numbers and performing the following simplecomparison: IF(colour0<colour1) use ramp 0 ELSE use ramp 1. Finally,exact color derivations are defined in the sections on each datastructure below.

For example, in the partition with two subsets, four colors are storedat 14 bits of precision (RGB 5.5.4). Each texel or pixel index is 3bits, so the ramps have 8 positions. The ramp derivation is as follows:

   unsigned BYTE c[8][4];  if(colour0 > colour 1)  {   c[0] = colour0;  c[7] = colour1;   // Set alpha to 1.0   c[0][0] = 0xff;   c[7][0] =0xff;   for(i=0;i<4;i++)   {    c[1][i] = (6*c[0][i] + 1*c[7][i] + 3) /7;    c[2][i] = (5*c[0][i] + 2*c[7][i] + 3) / 7;    c[3][i] =(4*c[0][i] + 3*c[7][i] + 3) / 7;    c[4][i] = (3*c[0][i] + 4*c[7][i] +3) / 7;    c[5][i] = (2*c[0][i] + 5*c[7][i] + 3) / 7;    c[6][i] =(1*c[0][i] + 6*c[7][i] + 3) / 7;   } } else {  c[0] = colour0;  c[7] =colour1;  // Set alpha to 1.0  c[0][0] = 0xff;  c[7][0] = 0xff; for(i=0;i<4;i++)  {    c[1][i] = (5*c[0][i] + 1*c[7][i] + 2) / 6;   c[2][i] = (4*c[0][i] + 2*c[7][i] + 2) / 6;    c[3][i] = (3*c[0][i] +3*c[7][i] + 2) / 6;    c[4][i] = (2*c[0][i] + 4*c[7][i] + 2) / 6;   c[5][i] = (1*c[0][i] + 5*c[7][i] + 2) / 6;  }    // Colour 6 istreated as transparent    c[6] = transparent; c[6][0] = 0; }

In the above example, each pixel in a subset is represented by a 3-bitindex into the color set.

In the partition with three subsets, alternatively, six colors may bestored at 20 bits of precision (RGB 7.7.6). Each texel index is 2 bits,so the ramps have 4 positions. The ramp derivation is as follows:

  unsigned BYTE c[4][4]; if colour0 > colour1) {  c[0] = colour0;  c[4]= colour1;  // Set alpha to 1.0  c[0][0] = 0xff;  c[4][0] = 0xff; for(i=0;i<4;i++)  {   c[1][i] = 2*c[0][i] + 1*c[4][i] + 1) / 3;  c[2][i] = 1*c[0][i] + 2*c[4][i] + 1) / 3;  } } else {  c[0] = colour0; c[4] = colour1;  // Set alpha to 1.0  c[0][0] = 0xff;  c[4][0] = 0xff; for(i=0;i<4;i++)  {   c[1][i] = (c[0][i] + c[4][i] + 1) / 2;  }  //Colour 3 is treated as transparent  c[3] = transparent; c[3][0] = 0; }

In the above example, each pixel in the subset is represented by a 2-bitindex into the color set.

In the partition with four subsets, eight colors are stored at 15 bitsof precision (RGB 5.5.5). Each pixel is represented by a 2 bit indexinto the color set. The color derivation is the same as for the 3partition mode. Similarly, in the partition with five subsets, tencolors are stored at 12 bits of precision (RGB 4.4.4) and the texelrepresentations work as in the four subset mode.

It should be noted that the optimal set of partitions for the formatwill be determined by the user and the nature of the application. Oneskilled in the art will recognize that it is impossible for one set ofpartitions to be totally optimal across all images.

Image Quality Versus DXT1

The above described first embodiment was tested against S3 Graphics'DXT1 (sometimes also called DXTn or DXTC) using the 64 partition setdescribed in FIG. 6. The tests revealed that this embodiment gives ahigher quality compression than DXT1 on all images tested, both in termsof measured RMS error and perceived image quality. Taking the Root MeanSquare (RMS) error as the basic quality parameter, the gains achievedrange from a typical low-bar of 10-15% reduction of total RMS error to ahigh range of 50% reduction or more.

The low-range RMS improvements are typically found on morephotorealistic textures, but even in cases where there is a relativelysmall improvement in overall RMS, the above described embodiment cangive significantly better perceived quality as it improves on some ofDXT1s most noticeable quality problems, noticeably its tendenciestowards introducing low-frequency noise (or ‘blocking’) and bleedingcolors from one region to another. The largest improvements in RMS tendto occur on non-photorealistic textures such as cartoons, or items like“heads-up displays,” where the above described embodiment usually givesa very significant improvement in visual quality.

The variable partition scheme described above is superior in terms ofRMS error. Additionally, it also provides noticeable improvement inperceived quality and eliminates some image artifacts almost completely.The table below illustrates some test results for various types ofimages.

Weighted RMS Error R = 0.3086; G = 0.6094, B = 0.082 Image Name ImageDescription DXT1 4bpp partition 4.2.03 Photographic; noisy 5.97 2.86 17Computer display 7.28 5.6 Dialog 1 Cartoon 5.55 3.28 Lena Photographicportrait 3.74 2.82 Ring Artificial, concentric 12.71 6.29 gradients withvarying frequency Smart Photographic, complex 7.8 5.57 Color regions

Image Quality Versus Other Compression Methods

In testing, the above described embodiment consistently produces higherquality images than the other compression methods discussed in theintroduction to this document, such as palletisation and vectorquantization. Comparing schemes at the same compression rate has shownthe above embodiment to be of higher quality in terms of RMS error andsubjective quality than other fixed-rate compression formats. It alsoadapts very well to a wide variety of input image types, and largelydoesn't depend on one particular type of input data (e.g., photographic)to produce high quality compression-some of other compression methodsdiscussed perform reasonably well on a subset of images, but break downwhen given a wider range of data.

Alternative Embodiments

A second embodiment is intended as a direct replacement for DXT5. Itsdata structure is similar to the first embodiment described above andDXT1. It uses the same color encoding as the first embodiment, but each8×8 color block is accompanied by an 8×8 alpha block with the followingformat:

struct {  unsigned int alpha0:8;  unsigned int alpha1:8;  unsigned intalpha2:8;  unsigned int alpha3:8;  unsigned int alpha4:8;  unsigned intalpha5:8;  unsigned int alpha6:8;  unsigned int alpha7:8;  unsigned intt00:3, t01:3, t02:3, t03:3, t04:3, t05:3, t06:3, t07:3;  unsigned intt08:3, t09:3, t0a:3, t0b:3, t0c:3, t0d:3, t0e:3, t0f:3;  unsigned intt10:3, t11:3, t12:3, t13:3, t14:3, t15:3, t16:3, t17:3;  unsigned intt18:3, t19:3, t1a:3, t1b:3, t1c:3, t1d:3, t1e:3, t1f:3;  unsigned intt20:3, t21:3, t22:3, t23:3, t24:3, t25:3, t26:3, t27:3;  unsigned intt28:3, t29:3, t2a:3, t2b:3, t2c:3, t2d:3, t2e:3, t2f:3;  unsigned intt30:3, t31:3, t32:3, t33:3, t34:3, t35:3, t36:3, t37:3;  unsigned intt38:3, t39:3, t3a:3, t3b:3, t3c:3, t3d:3, t3e:3, t3f:3; }APC2_ALPHA_BLOCK;

The alpha block is subdivided into subsets as with the color block, butthere are no explicit partitioning bits used in the format. Instead, theendpoints are ordered for each subset in the partition to derive the 4bits chosen from a table of 16 possible partitions.

For alpha encoding, a lower number of subsets per partitions areacceptable because the quality gains from additional subsets rapidlyreaches diminishing returns due to the generally high quality of thebasic compression scheme. Avoiding explicit partition bits allows theprecision of the endpoints to be kept as high as in DXT5. The6-interpolant encoding with explicit 0 and 1 is no longer used, but thisloss is generally more than offset by the addition of pre-compressionpartitioning. Alpha derivation should typically have at least 12 bits offractional precision retained. Although the old DXT5 only required8-bits of precision for the derivation, it is more flexible to allow theformat to use the full potential precision of the interpolated values.

For the color block, decoding in the above described formats typicallymeans that transparent punch-through alpha encoding is no longerrequired. However, the ability to decode to either 3 or 4 colors foreach subset in the partition could make a quality difference. Asdetailed below, there are a number of possible extensions to this secondembodiment format.

One example is an alpha extension. As noted above, the number ofpartitions for the color block for the first embodiment was limited toonly 64 possibilities, leaving only two “spare” bits. In the format ofthis second embodiment, one additional possibility is to make use ofthese two bits to decide on a per-block basis which channel isrepresented in the alpha block. Given the two bits we have fourpossibilities—

0—Colour block contains RGB, Alpha block contains A

1—Colour Block contains AGB, Alpha block contains R

2—Colour block contains RAB, Alpha block contains G

3—Colour block contains RGA, Alpha block contains B

After decoding the channels would be swizzled back into the normalorder. By selecting different swizzles for each block, significantimprovements in compression quality are possible.

Third through sixth embodiments differ from the above described firstembodiment in that they compress a different number of components—i.e.,instead of compressing three color components in RGB space, theycompress a single component or other texture variable. A thirdembodiment, for example, is a 1-component compressed format for singlechannel data. It uses the same compression as the alpha block in thesecond embodiment and allows compression of original data with around12-bits of precision to 4-bits.

A fourth embodiment is a 2-component compressed format. It is thesubstantially similar to ATI2N/3DC compression (developed by ATITechnologies), and uses the same block format for each component as theabove described third embodiment.

A fifth embodiment is a 4-component compressed format using the sameblock format for each component as the third embodiment.

A sixth embodiment format is designed for developers who require higherquality compression than that provided by the above described firstembodiment. It compresses to 8-bits per pixel (compared to the firstembodiment at 4-bits per pixel), but the compressed texture quality ismuch higher, and the format can handle images with 3 or 4 channels.Textures compressed with this sixth embodiment are generally nearlyindistinguishable from the uncompressed source texture. The principlesof the sixth embodiment are very similar to the first embodiment, butworks by compressing 4×4 pixel blocks.

In the sixth embodiment, each block contains either two or three subsetsper partition.

    typedef union {  struct  {   unsigned int blockType : 1;   unsignedint colour0:19;   unsigned int t0:3;   unsigned int t1:3;   unsigned intt2:3;   unsigned int t3:3;   unsigned int partitionBit0:1;   unsignedint colour1:19;   unsigned int t4:3;   unsigned int t5:3;   unsigned intt6:3;   unsigned int t7:3;   unsigned int partitionBit1:1;   unsignedint colour2:19;   unsigned int t8:3;   unsigned int t9:3;   unsigned inttA:3;   unsigned int tB:3;   unsigned int partitionBit2:1;   unsignedint colour3:19;   unsigned int tC:3;   unsigned int tD:3;   unsigned inttE:3;   unsigned int tF:3;  } twoPartitionMode;  struct  {   unsignedint blockType : 1;   unsigned int colour0:13;   unsigned int t0:3;  unsigned int t1:3;   unsigned int t2:3;   unsigned int t3:3;  unsigned int t4:3;   unsigned int t5:3;   unsigned int colour1:13;  unsigned int colour2:13;   unsigned int t6:3;   unsigned int t7:3;  unsigned int colour3:13;   unsigned int colour4:13;   unsigned intt8:3;   unsigned int t9:3;   unsigned int colour5:13;   unsigned inttA:3;   unsigned int tB:3;   unsigned int tC:3;   unsigned int tD:3;  unsigned int tE:3;   unsigned int tF:3;   unsigned int partitionBit:1; } threePartitionMode;  DWORD rawData[4]; } OPC8_BLOCK;

The <blockType> specifies if the data structure contains two or threesubsets per partition. There is no transparency encoding, and the colorramps always have 8 points. As illustrated, the sixth embodiment has anumber of different partitions.

For a two subset partition, the data structure contains four endpoints,specified at 19 bits (RGB 6.7.6) precision. The index size is three bitsand there are 32 possible partitions, selected in some fashion similarto the following:

partition=(block->colour0>block->colour1) ? 0x1: 0;

partition|=(block->colour2>block->colour3) ? 0x2: 0;

partition|=(block->partitionBit0) ? 0x4: 0;

partition|=(block->partitionBit1) ? 0x8: 0;

partition|=(block->partitionBit2) ? 0x10: 0;

For a three subset partition, the data structure contains six endpoints,specified at 13 bits (RGB 4.5.4) precision. The index size is threebits. There are 16 possible block partitions, selected in some fashionsimilar to the following:

partition=(block->partitionBit) ? 0x1: 0;

partition=(block->colour0>block->colour1) ? 0x2: 0;

partition|=(block->colour2>block->colour3) ? 0x4: 0;

partition|=(block->colour4>block->colour5) ? 0x8: 0;

An implementation of the sixth embodiment has been tested and shown togive quality levels that on most 3-channel (RGB) textures can beconsidered ‘perceptually lossless.’ In other words, if the compressedand uncompressed representations are compared side-by-side, it can bedifficult for an observer to determine which is which, even whenmagnified and subjected to close scrutiny. Preservation of detail andcolor is extremely good, and noticeable artifacts are very rare. Thecompression quality is high on both real-world and artificial images,when comparing the sixth embodiment to DXT1, RMS error on the 3-channeltextures is typically reduced by at least 50%, and more commonly by 70%or more. While normal map—i.e., a map of surface normals-compression canalso potentially be achieved with this format, the quality is not ashigh as other alternatives (3DC/BC5).

CONCLUSION

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. Note that numerous features describedabove can be implemented in data compression schemes outside the fieldof texture compression. It will be apparent to persons skilled in therelevant art that various changes in form and detail can be made thereinwithout departing from the spirit and scope of the invention. Thus, thebreadth and scope of the present invention should not be limited by anyof the above-described exemplary embodiments, but should be defined onlyin accordance with the following claims and their equivalents.

What is claimed is:
 1. A method for performing texture compressioncomprising: splitting an original texture into one or more originalpartitions of pixels, wherein the size of the original partitions ofpixels is variable; selecting one or more subsets within each of the oneor more original partitions of pixels that defines a further split foreach of the original partitions of pixels, wherein each of the subsetsdefine a plurality of disjoint subsets of pixels that comprise eachoriginal partition of pixels; and individually compressing each subsetwithin each original partition of pixels to form a compressed textureblock.
 2. The method of claim 1, wherein the size and arrangement of theone or more subsets within each of the one or more original partitionsare defined by a number of bits and selected from a predefined set ofpartitions.
 3. The method of claim 1, wherein the size and arrangementof the one or more subsets within each of the one or more originalpartitions of pixels is variable.
 4. The method of claim 1, wherein thesize of the one or more subsets within each of the one or more originalpartitions of pixels are different.
 5. The method of claim 1, wherein aunion of the one or more original partitions of pixels is the originaltexture.
 6. The method of claim 1, further comprising: decompressingeach compressed texture block to obtain an approximation of eachoriginal partition of pixels; comparing each original partition ofpixels to each approximation of the original partition of pixels; andobtaining a quality parameter for each compressed texture block.
 7. Themethod of claim 6, further comprising: selecting a final subset for eachof the one or more original partitions of pixels where the qualityparameter meets a pre-selected threshold; and individually compressingeach final subset within each original partition of pixels to form afinal compressed texture block.
 8. The method of claim 7, wherein thefinal subset for each of the one or more original partitions of pixelsis selected for the original partition of pixels if the qualityparameter for an identified subset meets a pre-selected threshold andthe final subset is selected that either meets the pre-selectedthreshold or is closest to the pre-selected threshold when none of thesubsets meet the pre-selected threshold.
 9. The method of claim 1,wherein the individually compressing each subset within each originalpartition of pixels uses two explicit endpoint parameters and a numberof intermediate points defining a ramp in the color space.
 10. Themethod of claim 1, wherein the individually compressing each subsetwithin each original partition of pixels compresses each of the subsetsusing a palletization technique.
 11. A computer processing systemcomprising: a processor and a memory, wherein the processor isconfigured to: split an original texture into one or more originalpartitions of pixels, wherein the size of the original partitions ofpixels is variable; select one or more subsets within each of the one ormore original partitions of pixels that defines a further split for eachof the original partitions of pixels, wherein each of the subsets definea plurality of disjoint subsets of pixels that comprise each originalpartition of pixels; and individually compressing each subset withineach original partition of pixels to form a compressed texture block.12. The computer processing system of claim 11, wherein the size andarrangement of the one or more subsets within each of the one or moreoriginal partitions are defined by a number of bits and selected from apredefined set of partitions.
 13. The computer processing system ofclaim 11, wherein the size and arrangement of the one or more subsetswithin each of the one or more original partitions of pixels isvariable.
 14. The computer processing system of claim 11, wherein thesize of the one or more subsets within each of the one or more originalpartitions of pixels are different.
 15. The computer processing systemof claim 11, wherein a union of the one or more original partitions ofpixels is the original texture.
 16. The computer processing system ofclaim 11, wherein the processor is further configured to: decompresseach compressed texture block to obtain an approximation of eachoriginal partition of pixels; compare each original partition of pixelsto each approximation of the original partition of pixels; and obtain aquality parameter for each compressed texture block.
 17. The computerprocessing system of claim 16, wherein the processor is furtherconfigured to: select a final subset for each of the one or moreoriginal partitions of pixels where the quality parameter meets apre-selected threshold; and individually compress each final subsetwithin each original partition of pixels to form a final compressedtexture block.
 18. The computer processing system of claim 17, whereinthe final subset for each of the one or more original partitions ofpixels is selected for the original partition of pixels if the qualityparameter for an identified subset meets a pre-selected threshold andthe final subset is selected that either meets the pre-selectedthreshold or is closest to the pre-selected threshold when none of thesubsets meet the preselected threshold.
 19. The computer processingsystem of claim 11, wherein the processor uses two explicit endpointparameters and a number of intermediate points defining a ramp in thecolor space when individually compressing each subset within eachoriginal partition of pixels.
 20. The computer processing system ofclaim 11, wherein the processor is configured to compress each of thesubsets using a palletization technique when individually compressingeach subset within each original partition of pixels.